The present invention relates to an image data conversion device and method for converting image data in accordance with a predetermined format.
Subject images captured on CCDs in still-video cameras are stored in digital form into storage devices such as memory cards, magnetic disks, or optical disks. In this case, each image involves a very large amount of data. For this reason, in order to record as many picture frames as possible in a limited storage capacity, it is required to perform some efficient compression on each digital picture signal.
As an efficient image data compression scheme, coding techniques are widely known that employ orthogonal transform coding and variable-length coding in combination. A typical technique has been set as an ISO (International Standards Organization) standard for compressing still images. This technique is summarized below.
First, image data is divided into blocks of a given size, and each block is subjected to two-dimensional discrete cosine transform (DCT) as orthogonal transform. Subsequent to the DCT, linear quantization is performed for each frequency component and Huffman coding is then performed on quantized values as variable-length coding. In this case, with DC coefficients, the differentials of DC coefficients in adjacent blocks are subjected to Huffman coding. With AC coefficients, on the other hand, a scan, called the zig-zag scan, is made from low-frequency to high-frequency components and two-dimensional Huffman coding is then performed on the number of consecutive zero AC coefficients (0 in value) and the value of the next nonzero AC coefficient.
This operation will be described more specifically with reference to FIG. 6. First, a frame of image data is divided into blocks of a given size (for example, blocks A, B, C, . . . of 8.times.8 pixels) as shown at (a) in FIG. 6. Two-dimensional discrete cosine transform (DCT) is then performed on each block as orthogonal transform as shown at (b). The resultant data are stored in an 8.times.8 matrix. Image data, viewed in a two-dimensional plane, has spatial frequencies which are frequency information based on the distribution of grayscale information. Thus, image data is transformed into a DC coefficient and AC coefficients by the DCT transformation. In the 8.times.8 matrix, data indicating the value (coefficient) of the DC component is stored in the origin (0, 0), data indicating the value of the highest-frequency AC component in the direction of abscissa is stored in the (0, 7) position, data indicating the value of the highest-frequency AC component in the direction of ordinate is stored in the (7, 0) position, and data indicating the value of the highest-frequency AC component in the diagonal direction is stored in the (7, 7) position. In the intermediate positions are stored the values of AC components which correspond with their respective coordinate positions with respect to the origin so that the further away from the origin, the higher the AC component frequency increases. The stored data in each coordinate position in the matrix is next divided by a quantization width for the corresponding frequency component to perform linear quantization on each frequency component (c). The Huffman coding is performed as variable-length coding on the resulting quantized values. For DC components, at this point, a process of taking the differential of DC components in adjacent blocks (hereinafter referred to as DPCM (Differential Pulse Code Modulation)) is performed and the resulting differential is represented by diffDC (d1). The diffDC is expressed by a group number (the number of additional bits) and additional bits that represents the amplitude of diffDC. The group number is Huffman coded to obtain a codeword. The codeword and the additional bits are combined as coded data (e1). The coded data is recorded on a storage medium such as memory card (f).
For AC components as well, each effective (nonzero) AC coefficient is expressed by a group number and additional bits that represent the value of the AC coefficient. For this reason, a zig-zag scan of AC coefficients is made from low-frequency to high-frequency components. Two-dimensional Huffman coding is performed on the number of consecutive zero AC coefficients (zero run length) and the group number of the value of the nonzero AC coefficient that follows the zero AC coefficients to obtain a codeword. The codeword and the additional bits representing the value of the nonzero AC component are combined as coded data (d2, e2). In the Huffman coding, the frequency of occurrence of each of DC and AC components in a frame of picture is estimated and bit assignment is made such that a component whose frequency of occurrence is closer to its peak is assigned a smaller number of bits. And a codeword is provided for each component. The above is the basis of the Huffman coding scheme.
An image data recording scheme having a hierarchical structure and a tile structure which is suited for editing and retention of digital image data on a computer and communications, i.e., a hierarchical data recording scheme has been proposed. As shown in FIG. 7A, this scheme has a feature in that it has a hierarchical structure of images with different resolutions. When an image has H.times.V pixels, images are also included which are 1/2, 1/4/ 1/8, . . . in the number of pixels. According to this scheme, any image can be fetched which has as many pixels as needed for an application. Since it is not required to open the image of the maximum resolution, the processing speed can be improved. Another feature is that each image is divided into tiles of 64.times.64 pixels as shown in FIG. 7B. In this system, even in the case where a high-resolution image (H.times.V pixels in FIG. 7B) is needed, only tiles needed for a present process can be fetched without opening the entire image, thus improving processing speed.
Such a recording scheme has an advantage that an image or a portion of an image which has a resolution suitable for an application can be fetched quickly to meet rapid display and editing requirements.
As described previously, a high-efficiency coding scheme has been proposed for the purpose of storing image data which allows image data to be compressed in an efficient manner, and moreover an image recording scheme has been proposed which is suitable for displaying, editing and processing image data. However, these schemes are not compatible with each other because they are different in image data format. In order to fetch compressed image data from a recording device and manipulate it in a format suitable for editing, a format change is required. In decoding compressed image data to original image information in accordance with the normal procedure and creating hierarchical data from the original image information, complicated, lengthy computations are involved, and moreover the image quality degrades.